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ABSTRACT 

Design and implementation of a curriculum evaluation 
plan for the vocational education program (occupational education 
under the Carl D. Perkins Vocational and Applied Technology Act) of a 
large correctional system are described. An outside evaluator 
assisted in designing and conducting the first year's evaluation. The 
1991-92 curriculum used was a modification of the Massachusetts 
Competency Based Vocational Educational Curriculum encompassing 16 
occupational areas and the academic competencies required* Curriculum 
treatment and its effects in year one were evaluated by experts in 
the vocational areas who observed the program, examined the 
curriculum, and interviewed instructors and a sample of students. An 
evaluation instrument was developed to provide uniformity of data 
collection across the 13 institutions studied. An evaluation design 
schedule was drawn up to form a time, task, and talent (TTT) chart 
for the first year. The TTT chart brings findings in the areas of 
student factors, program design, faculty, and facilities into a 
relationship with the personnel needed to carry out the evaluation 
and a time schedule that relates task completion to the use of the 
evaluation findings. The goal in years 2 and 3 will be to provide 
faculty and administration with information that will inform program 
decisions. Four charts present the evaluation design. An appendix 
contains the evaluation instrument. (SLD) 
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INTRODUCTION 



Curriculum evaluations on a large scale present a myriad of 
problems* Usually randomization is precluded by the scheduling and 
curriculum requirements of the subjects. The problems of 
specification of treatment variables and outcomes, as well as 
obtaining adequate measures and fair comparisons with equivalent 
populations, plague the curriculum evaluator methodologist , When 
curriculum evaluations are undertaken in the unusual settings of 
large correctional institutions, these methodological problems are 
further compounded by context variables unique to prison settings 
and disparate population samples. Prison populations have vastly 
over-representations of populations who: score low on basic 
achievement tests in mathematics and reading, are deficient on 
measures of social problem-solving skills, have no proficiency in 
vocational trade skills, and present scattered, uneven records of 
unskilled employment prior to incarceration. Social cognitive 
deficits which parallel these behavioral manifestations have been 
found to correlate highly with career criminal behavior patterns 
and subsequently a long history of involvement with the criminal 
justice system (Rose and Fabiano, 1985), While the consequences of 
these social cognitive deficits are readily apparent in the 
behavior of adjudicated youth within the setting of the 
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correctional institution, the measurement and evaluation of 
curricular and instructional efforts directed to remedy through 
fundamental change the deficits in the twelve social problem- 
solving skills common to these youth present unique methodological 
evaluation research problems. Previous research has identified 
twelve social problem-solving skills that youthful offenders lack 
or use ineffectively in their social functioning.: controlling 
first impulses, taking another^s perspective, clarifying and 
formulating the prcblem, setting appropriate and realistic goals, 
accurately appraising one*s competence in the situation, generating 
effective solutions, evaluating and anticipating consequences of a 
chosen solution, anticipating potential difficulties in carrying 
out a solution, and adjusting behavior by getting and using 
feedback (Larson, 1988). Occupational education, because it builds 
on direct behavioral assessment by the individual of actions and 
products, and fosters the exercise of talent, skills and emotions 
when engaged in a work setting, can contribute to personal 
understanding of self and employment skills (Lattimore, 1990; 
Hassell, 1988). The lack of either of these dimensions condemns 
the individual to social marginality as a citizen, employee and 
nuclear family member. 

How severe is the problem of imprisonment in the United 
States in general and in the Commonwealth of Massachusetts in 
particular? The United States ranks number one among nations in 
incarceration and in number of individuals involved with the 
criminal justice system at a rate of 400 per 100,000 of population. 
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For comparative purposes, The Netherlands rate is 21 per 100,000, 
HXid Great Britain is 80 per 100, 000, The Commonwealth of 
Massachusetts has a much lower rate than the United States in 
general with a rate of 175 per 100,000. Appalling as these base 
figures may seem, the equally, if not more troubling fact, is in 
the rate of increase, with 10% to 11% per year over the last four 
years (Imel, 1990). Crimes resulting in incarceration vary sharply 
among populations, especially by race, income and educational 
levels. A recent study in one major east coast city (Washington, 
D.C.) found 42% of black males age 18 to 35 enmeshed in the 
criminal justice system, either incarcerated (15%), on probation or 
parole (21%) , awaiting trial or being sought for arrest (6%) . And 
70% are arrested by the age of 35 and about 85% are arrested at 
some point in their lives (De Parle, 1992). Among white males in 
the same city, the estimates are that less than 10% were involved 
in any way with the criminal justice system, and a review of the 
population of those in detention found 98% black and 1% white. 
Black inner city youth who have been encountering greater problems 
with employment, education and family stability are swelling the 
ranks of the incarcerated at an increased rate over the last 
decade . 

The Nat ional Institute of Law Enforcement and Criminal 
Justice published a statistical profile of the United States' 
prison population that reflects the disadvantageous status this 
population occupies when compared to the normative status of the 
general populat ion . 

» 5 
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* Forty percent of all jail and prison inmates have 

completed high school compared to 85% of 20 to 29 year 
old males in the U.S. population. 

* The proportion of high school dropouts is three times 

larger among the incarcerated. 

^ Six percent of all inmates have no schooling at all and 
their rate of incarceration is three times higher than 
the next highest group - high school dropouts. 

* College graduates have an extremely low incarceration 

rate . 

* Offenders are predominantly male (96%) and 

disproportionately young (50% under 20 years of age), 
black (47%) , and unmarried (80%) as compared to the 
general population . 

Most offenders are poor. The average unemployment rate 
for offenders prior to arrest was about 40%. Of those 
who were employed prior to arrest, 80% made less than a 
poverty-level salary. Twelve percent of those who were 
employed worked only part time. 

* The typical female offender is under 30, a single mother 

with two or more children, economically dependent and 
troubled by physical and/or mental illness, as well as 
drug and/or alcohol dependency. 

* It has been estimated that up to 40% of the adults now 

incarcerated need special education services because of 
learning disabilities and other handicapping conditions. 

* A substantial number of state prison inmates have grown up 

in a s ingle parent household . 

(Massachusetts Department of Corrections, 1992) 

Public response to this growth in crime has been primarily 
directed toward time-honored controls on crime: building more 
detention facilities and imposing longer sentences. A rash of 
prison construction at an average cost of $50,000 per bed has been 
underway, but at this point there appears to be a consensus among 
criminal justice experts that prison facilities cannot be 
constructed at a rate that keeps pace with arrest by the police and 
sentencing to confinement by the courts, actions that are viewed as 
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favorable responses by the general public who seek public order and 
personal security (Lyall, 1992). A strong contributor to the 
swelling prison population has been the change in sentencing laws 
which mandates determinate sentences for specific crimes, 
particularly those against the person, and reduces the latitude for 
early parole granted in exercise of judicial judgment (Criver, 
1991). Though these structural changes have driven up the costs of 
the criminal justice system at a rapid rate, the public exhibits a 
failure to connect the actions at the front end of the criminal 
justice system to the high cost and low returns in the reciprocal 
results of limited deterrence of crime and high recidivism among 
offenders . 

Incarceration represents the most expensive form of 
correctional supervision and the resultant outcomes in reduction of 
crime have been found to be almost negligible (Commonwealth of 
Massachusetts, 1990). What seems to be lacking in this misplaced 
faith of the American people in longer sentences and securer 
prisons is that long sentences served with other anti-social 
individuals embeds in the individual a persona long recognized by 
those with extensive experience with the incarcerated as 
prisonizat ion , i.e., the inculcating of social norms and behaviors 
that increase the dysf unctionality of the individual in a normal 
community where trust must be won and choice exercised. In the 
enthusiasm for separating out and confining perpetrators of crime, 
the American public seems to have forgotten that nearly all 
pri- oners are released and rarely serve the total sentence (three- 



id 

ERLC 



Hi 



6 



fourths are paroled) , and most become eligible fc ; parole after 
serving one-third of their sentence, spending a proportion of time 
in incarceration and community-supervised time equal to about two- 
thirds of the maximum sentence adjudicated for the offense (Snarr, 
1985) , The offender who is convicted and sentenced (even life 
sentences in some cases become eligible for parole) serves only a 
fraction of his/her time in an incapacitation environment, i.e., 
segregated from the larger society, and in most cases is released 
within a one to seven year time frame after sentencing. Thus, 
despite public commitment to lock them up and leave them, the 
reality is that criminal offenders do return to society at a fairly 
rapid rate and confinement is a form of inconvenience rather than a 
period for reeducation or reintegration into society for severe 
offenders . 

A Bureau of Justice study of felons released and a six year 
follow-up found that almost 63% had been rearrested, 47% 
reconvicted, and 42% incarcerated within three years of release. 
The 63% rearrested had been charged with an average of five new 
offenses (Bureau of Justice Statistics, 1989). In short, 
recidivism is widespread and doing time seems to be of little value 
in redirecting the energies of offenders into positive, 
constructive behavior as opposed to the continuation of anti-social 
conduct that landed them in the criminal justice system originally. 
Correction education programs were designed to reintegrate 
offenders as useful citizens in society, able to support themselves 
and their families through productive employment and to function as 
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law-abiding citizens in the community in which they reside. As 
policy in the treatment of offenders has shifted, giving 
recognition to the fact that offenders will not spend the majority 
of their lives walled off from sc^iety, the interest and commitment 
to correction education has grown. Because of the basic education 
deficiencies of many prisoners in reading and mathematics skills 
(it is estimated that illiteracy afflicts from 50 to 75 percent of 
the prison population) , literacy education in correctional 
institutions receives major attention. In a similar manner, the 
vocational training program has been seen as vital to a correction 
education curriculum that will be contributory to the employability 
of the offender upon release into society. To counter recidivism 
and to improve the quality of life of the incarcerated, state and 
federal legislation has moved to provide funding to inmates' 
education through improved and expanded corrections curriculum. 

Recent federal legislation has prescribed the development of 
occupational education state programs for inmates held at various 
levels of security. These programs have been predicated upon 
recent studies that demonstrate that vocational education is a 
successful contributor to the rehabilitation of inmates. The 
curriculum organized in the correctional system to address these 
two major deficits was a modularized, job-specific, competency- 
based curriculum ordered in scope and sequence to meet the special 
problem of students who are frequently transferred from facility to 
facility to continue their education. The curriculum will be 
implemented over a three-year period of funding and will include 16 
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vocational areas in 13 correctional institutions (see Chart 1). An 
integrated approach in teaching academic skills in reading and 
mathematics will be used. An evaluation is required during all 
three year?; which will be based upon "... acquisition of 
c mpetencies, the integration of vocational and academic skills and 
licences or other credentials gained." (Massachusetts Department of 
Correction, Division of Inmate Training, 1991). 



CHART 1 



Programs Sponsored by Carl D- Perkins Vocational 
and Applied Technology Act 





AGENCY 


PROGRAM TITLE 


$ AWARD 


1. 


Barn!^ table 


We Iding 


$12, 800 


2. 


Berkshire 


Auto, Carpentry 


0 0,57-1 


3. 


Bristol 


Carpentry 


$ 6,400 


4. 


DYS 


Carpentry /Green /Ma son 




5. 


Franklin 


Food Management 


$ 5,638 


6. 


Hampden 


Spec. Pop. Coordinator 


$1 3, 000 


7. 


Hampshire 


Draft ing 


C 8,200 


8. 


MCI - Framingham 


Aesthetician 


$12, 900 


9. 


MCI - Framingham 


Desktop Publishing 


$ 7,500 


10. 


MCI - Framingham 


Manicurist 


$12 , 050 


11 . 


MCI - Walpole 


Comprehensive Learning Lab. 


$49, 882 


12 . 


Norfolk 


Graphic Arts/Printing 




13. 


Norfolk 


Small Engine 


$ 987 


14. 


Plymouth 


Printing 


$12, 595 


15. 


Suffolk 


Print ing/Graphics 


$ 7,860 


16. 


Worcester 


Electronics /Baking /Landscaping 


$ 8,000 
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METHOD 



An outside evaluator from a state university was commissioned 
to assist in designing the curriculum evaluation plan and to 
conduct the first year's evaluation. The curriculum being used 
during the 1991-92 school year is a slightly modified Massachusetts 
Competency Based Vocational Education Curriculum which encompasses 
both the specific occupational areas (N=16; and the academic 
competencies required. Two approaches to evaluation were 
undertaken during year one. The curriculum treatment and its 
effects in year one were evaluated using experts in the vocational 
areaii being taught who observed the program, examined the 
curriculum as taught, and interviewed the instructors and a sample 
of students. While they (the experts) did have the major 
responsibility for judging the curriculum processes and products, 
they conferred with the outside evaluator on selection of data 
sources and data gathering procedures. 

An evaluation instrument was developed to provide uniformity 
to data collection across sites • The instrument gathered data on 
program components of: student enrollment, program outline, program 
requirements, student selection, student records, faculty 
qualifications, curriculum guides and use, activities and 
individuals supported by federal monies, assessment procedures, 
integration of vocational and academic skills, student use of 
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assessment, instructor needs assessment, student completion, 
advisory committees, incentives for participation, and adequacy of 
facilities (See Appendix A) . The items on the instrument were 
developed from an evaluation instrument produced by the 
Correctional Education Association (U.S.A.) and from the report of 
performance audit of the educational programs in state correctional 
institutions conducted in 1990 (Commonwealth of Massachusetts, 
1990). Evaluators familiar with the occupational education fields 
and correctional education visited each of the sites, conducted the 
interviews, examined records and instructional materials, and 
surveyed facilities. An e\'aluation design schedule was drawn up 
into a Time, Task and Talent Chart (See Chox.'^t 2) for year 1 of the 
three year evaluation. 

RESULTS 

The results from the sixteen components of the occupational 
education program that came under the purview of the Carl D. 
Perkins Vocational and Applied Technology Act funding are reported 
under four sub-headings: student factors, program design, faculty, 
and facilities. All data from interviews were analyzed 
qualitatively cross-sect ionally among the institutions, as wel] as 
individually by institution. A decomposition analysis of program 
was done to isolate the effects of federal funding, inasmuch as 
this was only a small percent of the total program funding- In the 
decomposition analysis, direct tracing of expenditures is made to 
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CHART 2 



TIME, TASK, TALENT (TTT) 





EVALUATION 


DESIGN (YEAR 


1) 






TPSK/ 

man* 


APRIL 
1992 




JUNE 


JULY 


1992 




■■■'''■■■'Mis "^^ 
Faculty 
P.I/s 


■ Select arid 
develcp 
tests 


Gather ' 
archival 
data 


Examine 
preliin. 
results 


Exarnine 
prelim, 
results 


Develop 
test 

schedule 




Conpetencies 
Faculty 
P.I.'s 


Establish 
list of 
corpetencies 


.0 ■ ■ 


■■■::'r:*>''"^^ 

Select & 
test 

instnments 




Develop 
oonpetencies 
test schedule 




3 ii^^ » 
of 1 & 2 
Faculty 
P.I.'s 


Review 
onsite 
curriculum 
goals 


'0 ■ - 


Specify areas 
of 

integration 




........p...-...,. 

Anal^^e 
prelim, 
data 


'm'"" 

•h. 


Assess. 
Year 1 
P.I.'s 


Observatiofis 
and interviews 
cxisite 


■ . .■ 






■ ' "Seocxi'd OBS 
and interviews 


uT 


5 " -iRaKitttty " ' m V. 
Workshop 
Faculty 
P.I.^s 










Workshops 


■I- 


Credentiating 
Faculty 














Schedule 
P.I.'s 






Preljjninary 
Eval. design 
years 2-3 




' Final Eval. 
design 
years 2-3 




P.I.'s 








Prelim. 
Repc")rt 
Year 1 


" Final Report 
Year 1 


■ *^ 



IBEND: B = Beginning and end of activity; 0 = Conpletion of activity; 0 = Cn-goiag activity 
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connect the effect of the funding on the programs, e.g., in one 
program the Perkins funds were used to buy a curriculum package 
that was presented through a computer station and served as an 
individually paced tutorial program in literacy skills. 

Student Factors 

Records of students, faculty and student interviews serve as 
the primary data base for this area. Student enrollment is 
predicated on the classification testing data that is done at an 
inmate's entry into the system and on the counselor's and teacher's 
judgment of whether an inmate can benefit from the program. In 
most of the 16 programs there is far more demand than there are 
slots for students. Because of the high percentage of illiteracy 
in the inmate population, there is a great need for applicants to 
become qualified in the basic skills of reading and mathematics 
prior to entering the vocational programs. It is estimated that 
80% of the inmates are handicapped by being: mentally retarded, 
hard of hearing, deaf, speech impaired, health impaired, multi- 
handicapped or afflicted with assorted learning disabilities. With 
this special population there is a need for instructors to be 
versed in special education approaches, and with the lowest 20% of 
the inmate population, in pedagogical strategies specific to a 
functional curriculum (Williams, 1989). With over 50% of inmates 
lacking basic literacy skills, i.e., below seventh grade reading 
and mathematics, these individuals are restricted from entry into 
the vocational program, though most are in literacy education 
(Wof ford, 1986) . Motivating inmate enrollment in the vocational 
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programs is not an issue as days are deducted from a sentence for 
class attendance. However, bringing students to the skill level to 
profit from the vocational programs is the challenge (Hackett, 
1990). Nevertheless, it was found that all programs would need 
expansion to meet the current and anticipated demand if more 
inmates qualified for entry. At the present time the students who 
are entered in the vocational program are those with the highest 
reading and mathematics scores and having the fewest behavioral 
disorder handicaps. 

Inmates in a few of the correctional institutions are 
frequently transferred among institutions and this is done outside 
any concern for their participation in an educational program. 
Consequently, the running of a continuous program is Heavily 
dependent on records following a student which document student 
performance in sufficient detail that he can be entered at the 
appropriate level in the new institution. This appears to be a 
major weakness in the across institutions curriculum coordination. 
Most vocational programs are not duplicated in all institutions. 
Where programs lend themselves to definite skill level development, 
e.g., welding, the student can be credited with accomplishment of 
skills by level instead of course completion which would in some 
cases leaven the disruptiveness of abrupt transfer. However , even 
with this adjustment it is a problem that requires considerable 
study and the collection of an adequate data base on students if a 
policy that is supportive of the correction education program is to 
emerge and be applied consistently across institutions. 
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Because the funding of the vocational programs was less than 
one year underway, it was not possible to assess student completion 
rates. Completion rates are most probably being influenced by 
events that are extraneous to the program such as inmate transfer 
and availability of facilities and instructors. 

Program Design 

The 16 programs were found to have program outlines and 
written curricula. These follow a program design of competency- 
based curricula which has been established by the Vocational 
Education Division of the Massachusetts State Department of 
Education. Student assessment records do not reflect these 
competency-based curricula in great detail and as a result it is 
not possible to assess student progress and to have an informed 
account of the students' progress. Students were aware of informal 
assessment of their progress in all classes. And in some classes 
where there was a direct linkage of class work to a licensing 
requirement, as in the aesthetician course, students were very 
aware of requirements and their f ulf i llment . It appears that the 
close assessment of competency is related to programs that are 
linked to reentry jobs upon release. Where these linkages were 
strong and direct, especially with active advisory councils, the 
program was engaged in a vigorous effort of competency assessment. 
Where vocational programs were largely self-contained in the 
institutions with little or no linkage to reentry jobs, specific 
competency assessment on the part of instructors and competency 
attainment awareness on the part of participants was limited. Some 
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of the courses were using computer-assisted instruction and in the 
cases, specifically of the literacy programs, the assessment 
procedures were excellent and an integral part of the instructional 
program. 

The integration of academic skills, social skills and living 
skills in the vocational program was minimal in most programs. 
Academic skills were more often found to be taught in an applied 
context. But teaching to enhance and expand the use of basic 
skills was not a priority for most vocational instructors. Given 
the inmate population's extensive deficiencies in reading and 
mathematics, and the long history of school failure that 
accompanies most offenders, teaching within the context of a 
functional use of skills has been found to be more successful in 
advancing learning than the direct teaching in isolation 
(Jengeleski, 1988) There was some recognition of the importance 
of integrating the vocational and the academic skills by 
instructors in the interview, but little integration was observed 
in the program and instructors posited a need for assistance in 
advancing the use of an integrated curriculum in the vocational 
program. 

Inmates suffer distin t handicaps in social and living skills 
- certainly a major contributor to their present incarceration and 
a major cause of recidivism as the new releasee fails to gain a 
foothold in employment (Downing, 1987). Experiencing the inability 
to engage in social relationships in a workplace or to effect a 
positive social persona in an interview, the releasee falls back 
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into old patterns of dysfunctional social behavior and ^he 
antisocial network which produced the original adjudication of a 
sentence requiring incarceration. The relationship of social and 
living skills to vocational success is so tightly drawn in 
effecting reduction in recidivism that many correctional educators 
are convinced that vocational education pursued without benefit of 
these personal and social skills will not produce an increase in 
successful reentry of the parolee into the outside community and a 
corresponding decrease in the current 60% to 70% recidivism rate. 
In general, classes in vocational education were not sensitive to 
giving attention to pedagogical strategies which develop social 
skills, i.e., small group instruction in which experience is gained 
in building cooperative goal structures, obtaining cooperation of 
group participants, perspective taking, and building trust and 
reliability in working with others. No evidence was accrued that 
social skills were understood by instructors as essential knowledge 
if successful reentry of inmates into the community were to be 
achieved. In a similar vein, the lack of living skills being 
integrated in the vocational courses was marked and a serious 
deficiency in the curriculum. 

Faculty 

Faculty teaching qualifications were set by teacher 
certification standards. There were areas of competency where the 
faculty needed to be upgraded. These have been further identified 
in the section on program. In interviews the faculty of several of 
the 16 programs were aware of the need to upgrade the teaching 
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skills and to introduce newer instructional techniques into the 
classroom. A closer coordination of the teaching of literacy 
skills by computer aided programs and the vocational program was a 
major area for future curriculum improvement. The correctional 
education program design and its incorporation of the programs 
funded by the Carl D. Perkins Act require conducting a cooperative 
needs assessment that uses the faculty's contribution to strengthen 
the overall instructional treatment, if inmates needs for 
reeducation that will foster successful reentry into the community 
are to be met. Faculty would benefit from as well as contribute to 
a needs assessment, as they become sensitized to the deficiencies 
in the curriculum and are encoi. raged to project more specific 
measurable curriculum goals (Shapiro, 1987). Undoubtedly, faculty 
inservice will be a product of such a needs assessment. Faculty 
were found to be in need of assistance in the use of instructional 
methodologies oriented to the special populations encountered in 
the inmates. The assistance needed becomes even more pronounced as 
vocational programs are expanded and the number of severely 
handicapped population enrolled increases (Nelson, 1987) . 

Program linkage to reentry jobs in the community is critical 
if parolees are to make a transition from the vocational program to 
jobs (Schumaker, 1990) . Advisory committees composed of community 
representatives from the occupational field are essential to 
promoting linkages of the training in vocational courses in the 
institution and reentry jobs. Yet few of the programs which have 
advisory committees use them to provide these necessary linkages. 
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The one program which has had a proactive use of community advisory 
committees has been the most successful in job placement of 
parolees . 

Facilities 

The quality of the facilities varies in the vocational 
programs* In general they are derived from space that was formerly 
built for purposes other than educational. Faculty and 
administration do their best to achieve program accommodation to 
limited space, but they admit that program quality is vitiated by 
the physical facilities. One potential remedy lies in the 
extension of the vocational program into community work settings, a 
change that would accomplish the two goals of expanding the 
facilities at little cost and placing job training in a natural 
community setting for greater ease in the transfer of job skills on 
reentry of the parolee into the community (Miller, 1989). The 
needs assessment recommended in the earlier section should assess 
this dimension for the vocational education program. 

The Carl D. Perkins Act monies on evidence support important 
and critical program components. They represent a small but 
important percent of the budget. The majority of the monies has 
been spent for materials and curricular programs. Unquestionably, 
they have provided extensions of programs that were not being 
funded. Because the programs are new and just beginning to be 
implemented, it is not possible to render a substantive judgment on 
the worth of the program treatment in year 1. The elements of the 
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treatment do offer a number of suggestions for the evaluation 
design for the programs for years 2 and 3, These evaluation 
compo, ents are presented in Chart 3 as a Time, Task and Talent 
Chart (TTT) , which will be used in a planning meeting with the 
faculty from the 13 corrections institutions which will collaborate 
on the evaluation design which will be used to assess program 
treatment in years 2 and 3 of the Carl D. Perkins Act funding. 



CONCLUSION 



The TTT Chart was organized from the findings of the first 
year evaluation. It brings these findings into a relationship witli 
the personnel (talent) needed to carry out the evaluation tasks and 
a time schedule that relates the task completion to the use of the 
evaluation findings. The goal of the evaluation for years 2 and 3 
will be to provide faculty and administration with information that 
will inform program decisions. Although the evaluation design is 
to meet the requirements of federal legislation and therefore 
focuses on vocational education, it recognizes that vocational 
education is one part of a larger corrections education program 
directed toward reeducation of the inmate. Due to the formative 
nature of the evaluation design, it provides for several meetings 
during the year and will draw up findings at several points during 
the year as opposed to an end of year report. We believe this 



20 



approach has the merit of allowing faculty to make course 
corrections and to posit new hypotheses for testing in the program 
treatment. Our experience in these participant evaluation projects 
has been that parties become more knowledgeable as the treatment 
and evaluation proceed, and as experience informs the logic of the 
original evaluation design, old hypotheses are refined and new 
hypotheses are suggested. In addition, program treatment is 
greatly refined as faculty gather data and subject it to analysis. 
Thus the TTT Chart is a first approximation from the year 1 data 
that will be refined and reconfirmed in our meetings with the 
faculty. It will undergo further revision as data are gathered in 
the 2nd yoar. In short, as an evolving design, it is directed to 
teaching faculty and administrators who profit from the assessment 
as an integral part of the treatment plan. 

Evaluation designs for projects of this nature assume common 
characteristics as they address the problems generic to the natural 
settings where program treatment takes place. These common 
characteristics are: a) to identify the effects or variables in 
which the client and evaluator have an interest; b) to develop 
measurements or observations of these effects or variables and 
establish their relationships; c) develop comparisons with groups 
or data; and d) generate the specific variables and their 
relationships that are producing the effects and specification or 
generalization of the results to individuals and groups. The 
evaluation design for year 1 required the identification of program 
treatment in vocational education in the complex of programs that 
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is seen in corrections: literacy, basic skills, work programs, as 
well as vocational programs that receive specific funding from the 
federal government • Many of the programs are funded from the state 
or from institutionally derived funds, chiefly from the sale of 
products produced by prison industries . 

Program treatment was identified and the vocational program 
supported by the federal funds, only a small percent, less than 
10%, was disaggregated, i.e,, was funded in the program by the 
federal monies. These funded components were the effects to be 
studied in the evaluation. However, they were intertwined with the 
program treatment and could not be put to the usual evaluation 
tests of program treatments. For example, federal monies purchased 
a machine for the printing/graphic arts program in one institution. 
Without those monies, the* program would not have been possible, but 
they represented only 20% of the expenditure for the program for 
faculty, supplies, and support itemi. of space, heat, electricity. 
The monies were critical to starting a printing program. For 
evaluation purposes, this purchase of equipment was crucial to 
having a treatment effect. But to evaluate what the effects are in 
years 2 and 3 of the program treatment necessitates a broader 
assessment, especially where these program monies of the federal 
government were intermingled with the total program funding effort 
and collectively comprised the treatment. Only if a broader 
evaluation of the treatment is carried out can meaningful data for 
program improvement be obtained (McClintock, 1987), In sum, the 
effort of the federally sponsored program was disaggregated for 



CHARTS 

PROfiRAMS AND IISF OF PE RKINS ACTFUNfDS 



INSTITUTION 

Alpha & Omega 

Barnstable 

Berkshire 

Bristol 

Franklin 

Hampden-Springfield 

Hampshire 

Life Resources 

Norfolk 

Plymouth 

South Cove 



PROfiRAM RJNnS FXPKNDITURES 
Salaries and Instrucaonal Supplies 
Welding Course, Salaries 
Salary Instructor and Instructional Supplies 
Gercial Program Supplies 
Salary and Instructional Supplies 
Salary for Special Needs Coordinator 
Salary and Consumable Supplies 

Salary and Consumable Supplies 
Capital Equipment 

Salaries and Consumable Supplies and Assessment 
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accounting purposes but it will be incorporated into an assess »nt 
of the total treatment effect in the evaluation design for the 
second component measurement. Observation of effects was equally 
problematic if one attempted to maintain the separateness of the 
treatment variables related to the federal monies. Thus the same 
compromise was struck as in determining the effects of the progiam 
to be evaluated. Measurement and observations of variates follow 
conventions common to evaluation practice: faculty and staff were 
interviewed, students were observed and interviewed, curriculum 
materials and instructional products were examined. These measures 
and observations were greatly refined for evaluations in years 2 
and 3 and will be examined by the faculty and staff in the reedn 
assessment and workshops held by the evaluators based on year 1 
data. The third component of the evaluation design comparison 
relied heavily on evaluators' judgments based on experience with 
other educational programs. Evaluations comparisons in years 2 and 
3 will rely more on wi thin-group-comparisons using the data from 
year 1 and on more standard comparisons which are derived from a 
competency-based curriculum prescribed by the State Education 
Department. Other standardized instruments on classroom 
observations will be used for years 2 and 3. The findings will be 
subjected to a decomposition analysis to specify the variates which 
are most likely to be producing the effects and to target the 
program treatment variates influenced by input of the federal 
monies. Generalization will be based on a description of 
population and its comparability across institutions (tliere are 13 
institutions presenting 16 programs). The generalization qualities 
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of the evaluation are of lesser concern than the results of the 
program treatments, effort, effect, and efficiency on educating 
inmates in the specific vocational programs. The curriculum in the 
vocational programs is in an embryonic stage in year 1. The goal 
will be to assist the faculty through evaluative data to adopt and 
adapt curriculum to the special needs of the inmate population to 
further their reeducation effect reentry, reduce recidivism, and 
promote their adjustment to the community. The test of the 
curriculum will be its congruence in treatment affect to this 
general guiding goal. The evaluation design for years 2 and 3 as 
proposed (see Chart 3) is to further the faculty's curriculum 
development effort to foster greater congruence of the operational 
curriculum and the general guiding goals of correction education. 
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CHART 4 

FV A1 .1 lATlON r>P<;inN AND AmVITIES 
(September 1992 - February 1994) 

SEPTEMBER 1992 FEBRUARY 1993 SEPTEMBER 1993 FEBRUARY 1994 



Needs 
Assessment 



Eval.. Class. 
Staff/Faculty 



IParllclpatory 
IFvaluatlon Design 



Student Entry 
to Program 



Classification iBegIn Record on 
Staff/Faculty Student Entry 



Student Assig. 
to Program 



Classification ICrlterla on Student 
fitatf/Faculty Assig nment Devel 



4 IStudent 
Transfers 



Classification 
Staff/Faculty 



Midpoint Report 
Student Entry 



5 IProqram Linkage X O , 

to Community /Advlscry Ctee. Committee Assess. 

Evaluator lot Linkage 



6 I Integration of 
Acad. & Voc Ed 



Faculty & 
Evaluator 



a 



7 jintegraton of 
Social Skills 



8 [integration of 
Living Skills 



Faculty & 
Evaluator 



[bbs. of Integration 
Incurr. 



I Observation of 
Social Skills 



Linkage Effort 
Report 



Obs. of Integration 
r|curr> 



MidpoiritReporet Obs. of Integration 
n tnt g^g^ion llnfcurr. 



Faculty & 
Evaluator 



9 |lnstructional 
Processes 



a 



a 



[observation of 
Living Skills 



[observation ol 
llnstr. Processes 



D 



1 olComm. Reentry , ^ rr; 

Job Settings >^Advlsory Ctee. Plan lor Job 
& [faculty iPlacement _ 



Observation of 
Living Skills 



Admin, of Classroo 
Instruction 

-A 



Log On Job 
Placements 



1 1 [Functional 
iCurrlculum 



Faculty & 
Evaluator 



[Needs Assessment 
iFunct. Curric 



1 2 iFacilities and 
[Program 



Evaluator 



a 



13|Studenl Progress & 
Comprehensior 



-acuity & 
jator 



1 4 1 Interim & F 
Report 



inal 



Preliminary 
Evaluation Report 



Midpoint Report 
n Social Skills 



[observation of 
Social Skills 



Midpoint Report 
on Living Skills 



[observation of 
Living Skills 



Midpoint Report 
on Instruction 



[Admin, of Classroom 
Instruction 



MldpointReport on Review of Job 
Jo b Placements [Pl acements 



Pilot Test Report Implemeritatlon of 
on Funct. Curric. I Functional Curric. 



Review of Facilities 
& Curr iculum 



Student Progress 
.Report 



Midpoint Report on Student Progress 
Student Progress Report _ 



Midpoint Evaluation Interim Report 
Report 1 ^^P°^ I 



D t Beginning and end of activity Schedule 
<J r Completion of an activity 



l^x On-going Activity 
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* INFORMATIONAL FOOTNOTES ON TTT CHART S 



Needs assessment. Using the data gathered and analyzed in year 
1, a needs assessment will be conducted with the faculty, 
classification and counseling staff in the workshop. The 
needs assessment will examine the components 2 through 13 and 
c'raw up a participative evaluation plan for year 2 and 3. 
Priorities will be assessed and assigned, instruments 
identified and scheduled, and purposes for reports 
established in the needs assessment. 

Student entry. The criteria for student entry into the 16 
vocation a 1 programs will be examined and related to the 
potential student body as compared with the students now 
served. As one example, there is a need to respond to using 
criteria that will be more inclusive and that facilitate 
breaking the barriers of gender that now exist in the 
vocational program . 

Student assignment. Once in the vocational program the 
appropriateness of assignment to learning domains and the 
establishment of appropriate skill clusters in the learning 
domains, for example, what is basic and what is advanced mill 
carpentry. Moreover, once students are assigned by set 
criteria, what part does previous knowledge and skill play, 
hou do psychological and behavioral criteria interact in 
determining assignment to a vocational area? In the 1st year 
evaluation it was found that such criteria are now used as an 
informal screen by counselors, classification staff and 
faculty . 

Student transfers. It is felt that inmate transfers are 

disruptive to the educational process. This component of the 
evaluation will gather data on these transfers and their 
effect on student programs. 

Program Linkage to community. Because successful reentry of 
inmates to community is closely correlated with inmates' 
ability to obtain jobs, the evaluation of this component 
assumes priority. The role of advisory committees will 
change from one of being advisory only to becoming proactive 
in job formation and location. 

Integration of academics and vocational education. The effort 
to promote integration of academics, particularly basic 
skills, and vocational curriculum will be assessed througli 
observations of the classroom, interviews with faculty, and 
the examination of curriculum materials. 



26 



7. Integration of social skills. Despite the focus of this 

evaluation on vocational education programs, the success of 
inmates in becoming productive citizens is largely dependent 
on their ability to break away from habits and relationships 
that generate anti-social, illegal conduct. Vocational 
education offers opportunities for teaching social skills, 
though little was being done in deliberate, directed 
teaching, and faculty did not seem to be aware of how to 
integrate in classroom and clinical instruction the 
attitudinal components of social skills as they taught the 
conceptual subject matter of trades. In short, the 
fundamental social relationships which are significant to 
relating to others, whether at work, in a family or on any 
formal or informal level, are a critical deficiency in the 
inmate population. 

8. Integration of living skills. Living skills, defined as those 

skills needed by a person to maintain a state of independent 
functioning, are not mastered by many of the handicapped in 
the inmate population. Though not a focal concern in this 
evaluation, they are a central factor in determining whether 
an inmate will be able to function upon reentry to a 
community. There is some attention given to living skills in 
the basic education program, but the vocational programs have 
not moved to integrate these into their instruction. A more 
definitive assessment of inmates needing living skills will 
be part of the needs assessment, and a plan for integrating, 
as appropriate, in the vocational curriculum projected. 

9. Functional curriculum. If the needs assessment determines 

there exist sufficient numbers of inmates handicapped 
severely enough to be unable to profit from vocational 
courses now offered, faculty and staff will be assisted to 
explore a functional curriculum possibly connected to work 
assignments that teach trade skills. A pilot test of a 
functional curriculum is provided for if a population 
sufficient to sustain a curriculum is available. 

10. Instructional processes. The special characteristics of the 

population, handicapped, previous records of school and work 
failure place a burden on instructional processes. In the 
year 1 evaluation, there was evidence that vocational 
instructors desired assistance with special methods for 
inmate populations. One very serious problem in instruction 
springs from the non-English speaking, primarily Spanish 
language inmates. They lack basic education in the native 
language and few instructors have command of Spanish. There 
is a need for bilingual instruction and use of special 
methods with this population. A series of instruments for 
observing instruction and obta ining student assessment of 
instruction will be used with faculty to gather data and map 
instructional improvement strategies . 
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Community reentry job settings. The activities charted in 11 
will be coordinated with those in 5. Follow-up will be done 
by the Advisory Committee on inmate success in the reentry 
jobs. A formal program does not exist in moving inmates into 
job placement, and a schedule for gradation of inmate work 
responsibility has not been established. There are many 
barriers to inmate employment and in a time of economic 
tightening which the state has been experiencing for the past 
three years, the barriers become heightened. However, almost 
all inmates are released, and evidence indicates they serve 
about one-third of their adjudicated sentence, about seven 
years for serious crimes and as short as 11 months on average 
in some prisons for less serious crimes. Recidivism was 
found to run at 75% in one of the prisons among inmates who 
serve sentences on the average of 11 months. The chief 
variable in contributing to recidivism was unemployment, no 
stable job. The tie between providing the inmate with some 
vocational skills , integrated with a fundamental 
understanding of basic social and life skills which promote 
employability on release, appears to be the critical 
educational task of correction education. Successful 
community reentry is the litmus test of the correction 
education program, job placement success its most viable 
indicator . 

Facilities and program. The vocational programs are in general 
short of space which severely limits programs. There are 
some movements afoot to remedy the deficiencies by 
administrators and instructors. Progress will be monitored 
in this area. 

Student progress and completion. The factors that effect 
student completion of the vocational program are not well 
understood. Inmate motivation plays a large part in program 
discontinuance, but this appears to be more of a label than 
an explanation. Student progress is recorded, but it is done 
in a general and non-specific record. There is a need to 
disaggregate these data if they are to be used to improve 
program from a better understanding of student progress and 
continuation in the vocational programs • Faculty in the 
needs assessment will be looking at competencies measures and 
grade evaluations to use in their courses. 

Interim and final report. Evaluation reports will be rendered 
and their results will be used with faculty and staff four 
times over the two-year life of the project. These reports 
will reflect the 13 components of the project and be used to 
improve the vocational curriculum funded by the Carl D. 
Perkins Act monies. 
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MASSACHUSETTS DEPARTMENT OF CORRECTION 
DIVISION OF INMATE EDUCATION 



PROGRAM EVALUATION GUIDE (VOCATIONAL) 

RFP^ 920CCEVAL 



Prepared By: 

Southeastern Educational Research Co., Inc. 

Contact Liaison: 
Dr. Rcnzo Ricciuti 
30 Nash Road 

South Weynoouth, MA 02190 
(617) 335-7783 

May 4, 1992 



Date 



Name of Institution Nmtc of Evaluator 



Has the program ( _) been implemented and is it 

(Ntme of Progrtm) 

underway at tin^c of visit? How long has this program been in existence? 



2. How many students are enrolled? 



3 , Briefly describe the program. Upon completion, how many hours would a student have 
completed? 



4, How arc students selected fcH" the program and what arc the requirements (prerequisites) for 
their participation? 



5 . Arc there records on students and what is the nature of these records, e.g. test scores, 
grades or skills achieved? 



What arc the faculty qualifications who arc teaching in the program? 



Is therc a written curriculum and is it being used? 



What do the federal nrvonies specifically support in the budget? Inquire about how the 
dollar amount is being expended. 



How arc vocational competencies being assessed? How would faculty like them to be 
assessed? Would they participate in field testing a competency-based instrument? 



Is there any merger of vocational and academic skills in the vocational program? 



Arc students awarc of assessment of student work effOTt and progress in the vocational 
program? Does the program use the evaluation instrument developed by the Inmate 
Education Division? 



V 



12. 



What do instructors see as the primary need to make the program successful in 
students? 



13. Have any students completed the program? Have they achieved licensing status, if 
available? 



14. Is there any evidence of involvement of business and industry advisory teams? 



15. Are there any incentives for inmates to participate in the program? 



1 6, How adequate are the facilities for conducting the program? 



17. Any other observations or comments (by the staff). 
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